Image sensor, focus detection method and storage medium

ABSTRACT

An image sensor, comprising: a plurality of photo-diodes arranged divided in a specified pupil division direction, so that a pixel signal is generated by subjecting respective light flux, that passes through different exit pupil regions of an imaging optical systems for a single micro-lens, to photoelectric conversion, and a control circuit that implements an imaging mode for alternately and repeatedly executing a first imaging operation and a second imaging operation, wherein the first imaging operation combines pixel signals corresponding to the pupil division direction and generates and outputs a pixel signal for storage, and the second imaging operation generates and outputs a pixel signal corresponding to the pupil division direction, for focus detection.

CROSS-REFERENCE TO RELATED APPLICATIONS

Benefit is claimed, under 35 U.S.C. § 119, to the filing date of priorJapanese Patent Application No. 2018-133315 filed on Jul. 13, 2018. Thisapplication is expressly incorporated herein by reference. The scope ofthe present invention is not limited to any requirements of the specificembodiments described in the application.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to an image sensor, a focus detectionmethod, and a storage medium, and in detail relates to an image sensorhaving focus detection pixels that output a pixel signal for performingfocus adjustment using a phase difference AF method, and imaging pixelsthat output a pixel signal for live view image display and for imagestorage, and to a focus detection method and a storage medium.

2. Description of the Related Art

An imaging device, that has focus detection pixels arranged at someparts within a two dimensional array of imaging pixels, that forms asubject image that has been formed using a photographing optical system,and also performs focus detection for the photographing optical systemusing a phase difference AF method, is known. Also, this imaging deviceperforms live view display based on output from imaging pixels, andalso, if a release button is operated, generates image data for storagebased on output from the imaging pixels and stores this image data instorage medium.

As was described previously, focus detection pixels are arranged betweenimaging pixels, which means that in the case of generating an imagethere is a method of excluding output of the focus detection pixels, andonly selecting output of imaging pixels. For example, with the imagingdevice disclosed in Japanese patent laid-open No, 2015-0161906(hereafter referred to as “patent publication 1”), a first pixel rowthat includes a row of focus detection pixels, and a second pixel rowthat includes a row of imaging pixels and does not include a row offocus detection pixels, are provided, and readout of a focus detectionsignal from the first pixel row, and readout of an image signal from thesecond pixel row, are alternately and asynchronously performed. Also, inJapanese patent laid-open No. 2015-005853 (hereafter referred to as“patent publication 2”) it is proposed to switch between a method ofrespectively reading out divided pixels of a photodiode, and adding andreading out divided pixels of a photodiode, for every pixel row.

The technology disclosed in patent publication 1 and patent publication2 described above are both imaging control methods with which it hasbeen assumed that either a pixel signal is read out from focus detectionpixels during live view display, or live view is performed with shootingstandby in progress. At the time of continuous shooting for stillpicture shooting also, besides at the time of live view, it is necessaryto perform switching of imaging drive mode. In this case, since invalidframes (control period for making the timing for pixel readout andcharge reset the same as for the next imaging drive mode) are required,continuous shooting speed is lowered by the extent of the invalidframes. Accordingly, the technology disclosed in the previouslymentioned patent publications is not suitable for rapid continuousshooting.

SUMMARY OF THE INVENTION

The present invention provides an image sensor, focus detection device,imaging method, and focus detection method that are suitable for rapidcontinuous shooting of still pictures.

An image sensor of a first aspect of the present invention comprises aplurality of photo-diodes arranged divided in a specified pupil divisiondirection, so that a pixel signal is generated by subjecting respectivelight flux, that passes through different exit pupil regions of animaging optical system for a single micro-lens, to photoelectricconversion, and a control circuit that implements an imaging mode foralternately and repeatedly executing a first imaging operation and asecond imaging operation, wherein the first imaging operation combinespixel signals corresponding to the pupil division direction andgenerates and outputs a pixel signal for storage, and the second imagingoperation generates and outputs a pixel signal corresponding to thepupil division direction, for focus detection.

A focus adjustment method of a second aspect of the present invention isa focus detection method for a focus detection device having an imagesensor with a plurality of photo-diodes, arranged divided in a givenpupil division direction, that generate a pixel signal by subjectingrespective light flux, that passes through different exit pupil regionsof an imaging optical system for a single micro-lens, to photoelectricconversion, the focus adjustment method comprising: when the imagesensor is executing a first imaging operation, performing processing tostore still image data based on pixel signals output from the imagesensor; when the image sensor is executing a second imaging operation,performing processing for focus detection based on pixel signals outputfrom the image sensor; and setting an imaging mode, in the image sensor,for alternately and repeatedly executing the first imaging operation andthe second imaging operation.

A non-transitory computer-readable medium of a third aspect of thepresent invention, storing a processor executable code, which whenexecuted by at least one processor, performs a focus adjusting method,the processor being arranged within a focus detection device having animage sensor with a plurality of photo-diodes, arranged divided in agiven pupil division direction, that generate a pixel signal bysubjecting respective light flux, that passes through different exitpupil regions of an imaging optical system for a single micro-lens, tophotoelectric conversion, the focus adjusting method comprising: whenthe image sensor is executing a first imaging operation, performingprocessing to store still image data based on pixel signals output fromthe image sensor; when the image sensor is executing a second imagingoperation, performing processing for focus detection based on pixelsignals output from the image sensor; and setting an imaging mode, inthe image sensor, for alternately and repeatedly executing the firstimaging operation and the second imaging operation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram mainly showing the electrical structure of animaging device of one embodiment of the present invention.

FIG. 2 is a block diagram mainly showing the electrical structure of animage sensor of an imaging device of one embodiment of the presentinvention.

FIG. 3 is a block diagram showing the structure of pixels of an imagingdevice relating to one embodiment of the present invention.

FIG. 4 is a circuit diagram showing the electrical structure of a pixelof the imaging device of one embodiment of the present invention.

FIG. 5 is a timing chart showing an operational example of an imagesensor when performing an electronic shutter operation at the time offocus detection pixels priority mode, in the imaging device of oneembodiment of the present invention.

FIG. 6 is a timing chart showing an operational example of an imagesensor when performing pixel signal readout at the time of focusdetection pixels priority mode, in the imaging device of one embodimentof the present invention.

FIG. 7 is a timing chart showing a first operational example of an imagesensor when performing an electronic shutter operation at the time ofimaging pixels priority mode, in the imaging device of one embodiment ofthe present invention.

FIG. 8 is a timing chart showing a second operational example of animage sensor when performing an electronic shutter operation at the timeof imaging pixels priority mode, in the imaging device of one embodimentof the present invention.

FIG. 9 is a timing chart showing an operational example of an imagesensor when performing pixel signal readout at the time of imagingpixels priority mode, in the imaging device of one embodiment of thepresent invention.

FIG. 10 is a timing chart showing an operational example of an imagesensor when performing an electronic shutter operation at the time ofimage only mode, in the imaging device of one embodiment of the presentinvention.

FIG. 11 is a timing chart showing an operational example of an imagesensor when performing pixel signal readout at the time of image onlymode, in the imaging device of one embodiment of the present invention.

FIG. 12A and FIG. 12B are flowcharts showing operation of an imagingdevice of one embodiment of the present invention.

FIG. 13 is a timing chart showing operation at the time of still picturecontinuous shooting, in the imaging device of one embodiment of thepresent invention.

FIG. 14 is a drawing showing targets for pixel addition of the imagesensor, in the imaging device of one embodiment of the presentinvention.

FIG. 15 is a drawing showing pixel addition results of the image sensor,in the imaging device of one embodiment of the present invention.

FIG. 16 is a drawing showing another example of readout from the imagesensor, in the imaging device of one embodiment of the presentinvention.

FIG. 17 is a drawing showing another example of pixel addition resultsof the image sensor, in the imaging device of one embodiment of thepresent invention.

FIG. 18A and FIG. 18B are timing charts showing timing for readout ofstill picture frames and phase difference frames, in an imaging deviceof one embodiment of the present invention.

FIG. 19 is a timing chart showing an operational example of an imagesensor when performing an electronic shutter operation, with a modifiedexample of focus detection pixels priority mode, in the imaging deviceof one embodiment of the present invention.

FIG. 20 is a timing chart showing an operational example of an imagesensor when performing pixel read out of one among a pair of focusdetection pixels signals, with a modified example of focus detectionpixels priority mode, in the imaging device of one embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

An imaging device of one embodiment of the present invention will bedescribed in the following. This imaging device has an imaging section,with a subject image being converted to image data by this imagingsection, and the subject image being subjected to live view display on adisplay section arranged on the rear surface of the camera body based onthis converted image data. A photographer determines composition andphoto opportunity by looking at the live view display. At the time of arelease operation image data is stored in a storage medium. Image datathat has been stored in the storage medium can be subjected to playbackdisplay on the display section if playback mode is selected.

Also, the imaging section of the imaging device of this embodiment hasfocus detection pixels arranged at some parts within a two-dimensionalarray of imaging pixels, and a subject image that has been formed by aphotographing optical system is imaged. When shooting still picturesusing continuous shooting, exposure for phase difference detection isperformed between one actual exposure and another actual exposure forstill picture shooting (refer, for example, to FIG. 13). Aftercompletion of this exposure for phase difference detection, a signalfrom phase difference detection pixels (focus detection pixels) is readout (refer, for example, to S43 in FIG. 12B), correlation computation isperformed or focus deviation amounts are calculated using the pixelsignal that has been read out, and focusing of a focus lens is performed(refer, for example, to S45, S47 and S55 in FIG. 12B).

FIG. 1 is a block diagram showing one example of the structure of animaging device (specifically, a digital camera, for example) 1 thatincludes a focus detection device of one embodiment of the presentinvention. It should be noted that in FIG. 1 solid line arrows show flowof data, and dashed line arrows show flow of control signals.

An imaging device 1 comprises an interchangeable lens 100 and a camerabody 200. The interchangeable lens 100 is configured so that it ispossible to attach to the camera body 200. When the interchangeable lens100 is attached to the camera body 200, the interchangeable lens 100 andthe camera body 200 are connected so that communication is possiblebetween them. It should be noted that the imaging device 1 is notnecessarily a lens interchangeable imaging device. For example, theimaging device 1 may be a lens integrated imaging device. The imagingdevice may also be provided within a portable device, such as a smartphone.

The interchangeable lens 100 comprises an imaging optical system 102, adrive section 104, a lens CPU (Central Processing Unit) 106, and a lensside storage section (memory) 108. Here, each block of theinterchangeable lens 100 is configured using hardware, for example.However, some blocks do not have to be configured using hardware, andmay be configured using software. Also, each block of theinterchangeable lens 100 need not be configured using a single hardwareor software component, and may be configured using a plurality ofhardware or software components. Also, in a case where theinterchangeable lens and the camera body are integrated, the lens CPU106 and the CPU 212 may be configured as a single CPU.

The imaging optical system 102 is an optical system for imaging lightflux from a subject on to the image sensor 208 of the camera body 200.The imaging optical system 102 comprises a focus lens 102 a and anaperture 102 b. The focus lens 102 a is constructed so as to be able toadjust focal position of the imaging optical system 102 by moving in anoptical axis direction.

The aperture 102 b is arranged on the optical axis of the focus lens 102a. The opening diameter of the aperture 102 b is variable. The aperture102 b adjusts amount of light flux from a subject passing through thefocus lens 102 a that is incident on the image sensor 208. The drivesection 104 has a drive motor and drive circuitry etc., and drives thefocus lens 102 a and the aperture 102 b based on control signals outputfrom the lens CPU 106. Here, the imaging optical system 102 may beconfigured as a zoom lens. In this case, the drive section 104 may alsoperform zoom drive. There may also be a configuration where the userdrives the zoom lens manually.

The lens CPU 106 is a processor that includes a CPU and peripheralcircuitry for the CPU, and operates in accordance with programs storedin a lens side storage section 108. The lens CPU 106 is configured so asto be able to communicate with the CPU 212 of the camera body 200 via aninterface (I/F) 110. The lens CPU 106 controls the drive section 104 inaccordance with control signals of the CPU 212 of the camera body 200.Also, the lens CPU 106 transmits various information, such as aperturevalue (F value) of the aperture 102 b, and lens information stored inthe lens side storage section 108, to the CPU 212 via the I/F 110.

It should be noted that the lens CPU 106 is not necessarily configuredas a CPU. That is, functions that are the same as those of the lens CPU106 may also be implemented using a processor such as an ASIC(Application Specific Integrated Circuit) or FPGA (Field-ProgrammableGate Array) etc. Also, functions that are the same as those of the lensCPU 106 may also be implemented using software.

The lens side storage section 108 is an electrically rewritablenonvolatile memory, and stores information relating to theinterchangeable lens 100. Lens information includes, for example,information focal length information and aberration information of theimaging optical system 102.

The camera body 200 comprises a mechanical shutter 202, a drive section204, an operation section 206, the image sensor 208, a hand shakecorrection circuit 210, the CPU 212, an image processing circuit 214, animage compression and expansion section 216, a focus detection circuit218, an exposure control circuit 220, a display section 222, a bus 224,DRAM (Dynamic Random Access Memory) 226, a body side storage section(memory) 228, and a storage medium 230. Here, each block of the camerabody 200 is configured using hardware, for example. However, some blocksdo not have to be configured using hardware, and may be configured usingsoftware. Also, each block of the camera body 200 need not be configuredusing a single hardware or software component, and may be configuredusing a plurality of hardware or software components.

The mechanical shutter 202 has an opening and closing structure, andadjusts a time for which light flux from the subject is incident on theimage sensor 208 (exposure time of the image sensor 208). A focal planeshutter, for example, is adopted as the mechanical shutter 202. Besidesthis focal plane shutter, a lens shutter may be provided in aninterchangeable lens (lens barrel side). The drive section 204 drivesthe mechanical shutter 202 based on control signals from the CPU 212.The drive section 204 comprises an actuator that drives the mechanicalshutter 202, and drive circuitry etc. for this actuator, and performsopening and closing operations of the mechanical shutter 202.

The operation section 206 is an interface for inputting userinstructions to the imaging device 1, and includes various operationmembers such as various operation buttons like a power supply button, arelease button, a movie button, a playback button, a menu button, etc.and a touch panel etc. This operation section 206 detects operatingstates of the various operation members, and outputs signalsrepresenting detection results to the CPU 212.

The image sensor 208 has a pixel section 22 (refer to FIG. 2) with aplurality of imaging pixels arranged two-dimensionally. Imaging pixelsare constructed divided into a plurality of focus detection pixels,corresponding to a microlens L (refer to FIG. 3). The focus detectionpixels generate photoelectric conversion signals by respectivelysubjecting light flux, that passes through regions resulting fromsubjecting a plurality of exit pupils of an imaging lens 2, which is aphotographing optical system, to pupil-division, to photoelectricconversion. A plurality of focus detection pixels are arranged twodimensionally in the pixel section 22. While the image sensor 208 isconstructed as a single CMOS image sensor provided with a primary colorBayer array color filter, this structure is not limiting. The detailedstructure of the image sensor 208 will be described later using FIG. 2to FIG. 4.

The image sensor 208 is an image sensor having a plurality ofphoto-diodes arranged divided in a given pupil division direction, for asingle micro-lens, so that a pixel signal is generated by subjectingrespective light flux, that passes through different exit pupil regionsof an imaging optical system, to photoelectric conversion. The imagesensor 208 is also an image sensor having a pixel section in which aplurality of light receiving sections are arranged divided in a givenpupil division direction, for a single micro-lens, so that a pixelsignal is generated by subjecting respective light flux, that passesthrough different exit pupil regions of an imaging optical system, tophotoelectric conversion.

The image sensor 208 is arranged on the optical axis of the imagingoptical system 102, at a position that is behind the mechanical shutter202, and where light flux from a subject is formed into an image by theimaging optical system 102. The image sensor 208 images a subject andgenerates a pixel signal relating to the subject.

The image sensor 208 of this embodiment is capable of operating in afocus detection pixel priority mode (simple readout system) where firstreadout is performed based on control of the CPU 212 functioning as asystem control section, an imaging pixel priority mode (subtractionreadout system) where second readout is performed, and image only mode(readout system with no phase difference information) where thirdreadout is performed.

Taking an example where a single image pixel is divided into two focusdetection pixels A and B, for example, with focus detection pixelpriority mode (simple readout system) a pair of focus detection pixelsignals A and B are respectively output from the image sensor 208 as aresult of first readout. As pairs of focus detection pixels A and B,there are left opening pixels Gbl, Brl, Rl and Bl, and right openingpixels Gbr, Grr, Rr and Br. Also, as pairs of focus detection pixels Aand B there are top-opening pixels Gbt, Grt, Rt and Bt, andbottom-opening pixels Gbb, Grb, Rb and Gb

Also, with image pixel priority mode (subtractive readout system), oneof either of the pair of focus detection pixel signals A and B (here,for example, it is made the focus detection pixel signal A), and animage pixel signal (A+B) resulting from addition of the pair of focusdetection pixel signals A and B, are output from the image sensor 208using second readout. Further, in image only mode (readout systemwithout phase difference information), only an image pixel signal (A+B)resulting from having added a pair of focus detection pixel signals Aand B is output using third readout, and neither of the focus detectionpixel signals A and B are output. Detailed operation of the firstreadout, second readout, and third readout will be described later usingFIG. 5 to FIG. 11.

The image sensor 208 operates under a first imaging operation wherepixel signals corresponding to pupil-division direction are combined togenerate and output a pixel signal for storage (refer, for example, tothe third readout shown in FIG. 10 and FIG. 11), and under a secondimaging operation where pixel signals corresponding to pupil-divisionfor focus detection are generated and output (refer, for example, to thefirst readout shown in FIG. 5 and FIG. 6). Also, the image sensor 208(CPU 212) functions as a control circuit that executes an imaging modeto alternately and repeatedly execute a first imaging operation and asecond imaging operation (refer, for example, to FIG. 13, which will bedescribed later). The CPU 212 also functions as a control circuit thatsets an imaging mode, to alternately and repeatedly execute the firstimaging operation and the second imaging operation, in the image sensor.It should be noted that the control circuit may be constructed withinthe same chip integrally with the pixel section 22 (refer to FIG. 2)within the image sensor 208, may be provided within a separate chip tothe image sensor 208, and further some of the functions of the controlcircuit may be implemented by the CPU 212.

Also, with the above described second imaging operation, only pixelsignals corresponding to some pupil-division directions of the pluralityof photodiodes are output (refer, for example, to the second readoutshown in FIG. 7 to FIG. 9, and to FIG. 14, FIG. 16, and FIG. 17). Also,with the above described second imaging operation, all pixel signalscorresponding to the pupil-division direction of the plurality ofphotodiodes are output (refer, for example, to the first readout shownin FIG. 5 to FIG. 6, FIG. 14 and FIG. 15). Also, with the image sensor208, the image sensor has a plurality of imaging modes, and among theplurality of imaging modes, in the case where an imaging mode thatalternately and repeatedly executes the first imaging operation and thesecond imaging operation is set, the first imaging operation is executedbefore the second imaging operation (refer, for example, to FIG. 18A).

The image sensor includes a second imaging mode where a third imagingoperation is performed to combine pixel signals corresponding to pupildivision direction, and generate and output a pixel signal for display(refer, for example, to second readout in FIG. 7 to FIG. 9, and S5 inFIG. 12A). The CPU 212 (and image sensor 208) functions as a controlcircuit that, if still picture shooting has been instructed in a casewhere the image sensor is repeatedly executing imaging operations in thesecond imaging mode, sets an imaging mode to alternately and repeatedlyexecute the first imaging operation and the second imaging operation inthe image sensor (refer, for example, to S31 and S43 in FIG. 12B).

The image sensor includes a third imaging mode where the second imagingoperation and the third imaging operation are successively executed(refer, for example, to S5 and S17 in FIG. 12A). The CPU 212 functionsas a control circuit that, if still picture shooting has been instructedin a case where the image sensor is repeatedly executing imagingoperations in the third imaging mode, sets an imaging mode toalternately and repeatedly execute the first imaging operation and thesecond imaging operation in the image sensor (refer, for example, toFIG. 12A and S31 and S43 in FIG. 12B).

The hand shake correction circuit 210 moves the image sensor 208 indirections parallel to the light receiving surface of the image sensor,so as to suppress camera shake that has been generated in the camerabody 200. By moving the image sensor 208 so as to negate camera shakemovement, blurring of the subject image occurring in image data that isattributable to camera shake is suppressed. It should be noted that thecamera shake correction circuit may be provided in the interchangeablelens 100. A camera shake correction circuit in this case is configuredso as to move a camera shake correction optical system that is includedin the imaging optical system 102.

The CPU 212 is a processor that includes a CPU and peripheral circuitryfor the CPU, and performs overall control of the camera body 200 inaccordance with programs stored in a body side storage section 228. TheCPU 212 controls imaging operations (imaging drive mode, readout modeetc.) by the image sensor 208, for example. Also, the CPU 212 outputscontrol signals, for driving the focus lens 102 a, to the lens CPU 106,in accordance with focus state of the focus lens 102 a that has beendetected by the focus detection circuit 218. The CPU 212 also outputsexposure setting values that have been calculated by the exposurecontrol circuit 220 to the lens CPU 106 and the image sensor 208. Here,the CPU 212 is not necessarily configured as a CPU. Specifically,functions that are the same as those of the CPU 212 may also beimplemented using an ASIC or FPGA etc. Also, functions that are the sameas those of the CPU 212 may also be implemented using software.

The image processing circuit 214 applies various image processing topixel data. For example, at the time of still picture shooting, theimage processing circuit 214 applies image processing for still picturestorage to pixel data to generate still picture data. Similarly, at thetime of movie shooting, the image processing circuit 214 applies imageprocessing for movie storage to pixel data to generate movie data.Further, at the time of live view display the image processing circuit214 applies image processing for display to the pixel data to generatedisplay image data. The image processing circuit 214 functions as animage processing circuit that, when the image sensor is executing thefirst imaging operation (refer, for example, to third readout shown inFIG. 10 and FIG. 11), performs processing for storing still picture databased on pixel signals output from the image sensor.

The image compression and expansion section 216 has an image compressioncircuit and an image expansion circuit. At the time of image datastorage, the image compression and expansion section 216 compressesimage data that has been generated by the image processing circuit 214(still picture data or movie data). Also, at the time of image dataplayback, image data that is stored in the storage medium 230 in acompressed state is expanded.

The focus detection circuit 218 performs focus detection for the focuslens 102 a using a known phase difference method that uses focusdetection pixel data output from the focus detection pixels of the imagesensor 208 (refer, for example, to S7, S9 S19 and S21 in FIG. 12A, andS45 and S47 in FIG. 12B, etc., which will be described later). The focusdetection circuit 218 functions as a focus detection circuit that, whenthe image sensor is executing the second imaging operation (refer, forexample, to first readout shown in FIG. 5 and FIG. 6), performsprocessing for focus detection based on pixel signals output from theimage sensor.

The exposure control circuit 220 fulfills a function as a photometrysection, and calculates exposure setting values based on pixel data ofthe image sensor 208. This exposure control circuit 220 measures subjectbrightness from pixel data of the image sensor 208, and calculatesexposure setting values necessary to make brightness of the subject atthe time of shooting a correct value, from the subject brightness thathas been measured. Exposure setting values include opening amount of theaperture 102 b (aperture value) and exposure time of the image sensor208 (shooter speed).

The display section 222 has a display such as a liquid crystal displayor an organic EL display, and is arranged on a rear surface etc. of thecamera body 200, and functions as an electronic viewfinder. This displaysection 222 displays images in accordance with control by the CPU 212.The display section 222 is used in live view display and display ofalready stored images etc.

The bus 224 is connected to the image sensor 208, CPU 212, imageprocessing circuit 214, image compression and expansion section 216,focus detection circuit 218, exposure control circuit 220, displaysection 222, DRAM 226, body side storage section 228 and storage medium230, and operates as a transfer circuit for transferring various datathat has been generated by these blocks.

The DRAM 226 is an electrically rewritable volatile memory, andtemporarily stores various data such as pixel data output from the imagesensor 208, still picture data, movie data, display image data, andprocess data for the CPU 212 etc. It should be noted that it is alsopossible to use an SDRAM (synchronous dynamic random access memory) astemporary storage.

The body side storage section 228 is an electrically rewritablenon-volatile memory. The body side storage section 228 stores variousdata such as programs used by the CPU 212 and adjustment values for thecamera body 200 etc. The storage medium 230 is an electricallyrewritable non-volatile memory, and is built into the camera body 200 orconfigured to be loaded into the camera body 200. The storage medium 230stores image data for storage as an image file of a specified format. Itshould be noted that the DRAM 226, body side storage section 228, andstorage medium 230 may be respectively configured as a single memory, ormay be configured as a combination of a plurality of memories etc.

Next, the structure of the image sensor 208 will be described using FIG.2. The image sensor 208 has imaging pixels that have been divided into aplurality of focus detection pixels, and generates image pixel signaland focus detection pixel signals based on photoelectric conversionsignals that have been generated by subjecting light flux from focusdetection pixels to photoelectric conversion.

In the example shown in FIG. 2, the image sensor 208 comprises avertical scanning section 21, a pixel section 22, an analog processingsection 23, an ADC processing section 24, memory section 25, horizontalscanning section 26, output section 27, input section 28, and elementcontrol section 29.

Image pixels and a focus detection pixels are arranged in the pixelsection 22. Generation of the image pixel signals and focus detectionpixel signals by photoelectric conversion is performed by at least onesection among the vertical scanning section 21 to output section 27, andthe element control section 29 etc. The structure of each pixel arrangedin the pixel section 22 will be described later using FIG. 3. Also,electrical connections of the image pixels and the focus detectionpixels will be described later using FIG. 4.

The vertical scanning section 21 has a vertical scanning circuit, andperforms scanning in a vertical direction by successively selectingpixel rows (lines) in a horizontal direction within the pixel section22. This vertical scanning section 21 selects a particular line, andcontrols charge accumulation time of pixels (exposure time) byperforming resetting and transfer of each pixel of the line that hasbeen selected.

The analog processing section 23 has an analog processing circuit, andsubjects an analog pixel signal that has been read out from the pixelsection 22 to analog signal processing. This analog processing section23 includes, for example, a preamp that amplifies the pixel signal, anda correlated double sampling (CDS) circuit that subtracts reset noisefrom the pixel signal, etc.

The analog digital conversion processing section (ADC processingsection) 24 has an A/D conversion circuit, and converts the analog pixelsignal that has been output from the analog processing section 23 to adigital pixel signal. This ADC processing section 24 adopts a structure,such as exemplified by camera ADC, for example, whereby a pixel signalthat has been read out from the pixel section 22 is subjected to ADconversion by an analog to digital converter (ADC) for every line.

The memory section 25 has a memory, and is configured by an electricallyrewritable volatile memory circuit etc. that temporarily holds a pixelsignal that has been converted by the ADC processing section 24.

The horizontal scanning section 26 has a horizontal scanning circuit,and reads out pixel signals (image pixel signals and focus detectionpixel signals) from the memory section 25 in the order of columns.

The output section 27 has an output circuit, and organizes pixel signalsthat have been read out from the horizontal scanning section 26 forgenerating pixel signal rows, converts to an output signal format suchas a serial signal or differential signal etc. and outputs the convertedresult. It should be noted that this output section 27 or the abovedescribed ADC processing section 24 etc. function as a sensitizationsection that performs sensitization processing (signal amplificationprocessing in accordance with ISO sensitivity that has been set).

The input section 28 has an input circuit, and receives synchronizationsignals, a reference clock and operation setting information etc.relating to control of the image sensor 208 from a system controlsection 14.

The element control section 29 has an imaging control circuit, and isfor controlling each block within the image sensor 208 in conformitywith synchronization signals and a reference clock that have beenreceived via the input section 28, and is provided with a readout methodselection section 30. Also, the element control section 29 receivesoperation setting commands, such as commands for switching imaging drivemode, from the CPU 212 via the input section 28, and controls each blockwithin the image sensor 208.

The readout method selection section 30 has a selection circuit, andselects and sets a readout method for readout from the image sensor 208based on operation setting information (for example, camera modes suchas still picture shooting, movie shooting, live view, AF etc.) that hasbeen received via the input section 28. As readout methods, firstreadout (simple readout system), second readout (subtraction readoutsystem), and third readout (readout system without phase differenceinformation) etc. are provided. The element control section 29 controlseach section within the image sensor 208 in accordance with a readoutmethod that has been set by the readout method selection section 30.

A control section that controls readout of the image sensor isconfigured by the element control section 29 and the CPU 212 that wasshown in FIG. 1. The control section controls the image sensor 208service to perform first readout in a given frame, and perform secondreadout in another given frame. Also, the control section controls theimage sensor 208 so as to perform third readout in yet another givenframe. Also, the element control section 29 has a predetermined priorityof imaging drive modes, and for each single frame it is possible toexecute an imaging drive mode where any one of the first, second, orthird readout are executed, an imaging drive mode where a combination ofa plurality of readouts are executed, and further an imaging drive modewhere those operations are repeatedly executed etc. For example, as willbe described later, there are an imaging drive mode where still pictureframes (third readout) and phase difference frames (first readout)continue to be executed alternatively and repeatedly, and an imagingdrive mode where live view (display) frames (third readout) continue tobe executed repeatedly. In this way, the image sensor 208 is capable ofexecuting imaging operations in accordance with an imaging drive modethat has been instructed from the CPU 212.

First readout is generation and readout of both of a pair of focusdetection pixel signals for a first pupil-division direction, based onthe photoelectric conversion signal. Also, second readout is generationof one of a pair of focus detection pixel signals for a secondpupil-division direction based on a photoelectric conversion signal,together with generation of an image pixel signal by addition of allphotoelectric conversion signals that have been generated within asingle image pixel, and reading out the one focus detection pixel signalthat has been generated and the image pixel signal. Further, the thirdreadout is generation of an image pixel signal by addition of allphotoelectric conversion signals that have been generated within asingle image pixel, and reading out the image pixel signal that has beengenerated.

It should be noted that in FIG. 2 a structural example is shown in whichthe image sensor 208 is not only provided with the vertical scanningsection 21 and the pixel section 22, but is also provided with theanalog processing section 23-element control section 29. However, thestructure of the image sensor 208 is not thus limited, and it ispossible for one or more of the analog processing section 23-elementcontrol section 29, for example, to be arranged externally to the imagesensor 208.

Next, the structure of the focus detection pixels and image pixelsarranged in the pixel section 22 will be described using FIG. 3. As hasbeen described above, the pixel section 22 is a pixel array sectionhaving image pixels and focus detection pixels arranged twodimensionally (for example, in the vertical direction (column direction)and horizontal direction (line direction)).

FIG. 3 is a table showing an example of a pixel structure in which twoor four photodiodes PD are arranged in a single microlens L. In FIG. 3,a 2PD pixel structure and a 4PD pixel structure are exemplified asstructures for the image pixels. The 2PD pixel structure has twophotodiodes PD arranged for a single microlens L. The 4PD pixelstructure has four photodiodes PD arranged for a single microlens L.

Each pixel is configured with a microlens L, color filter F andphotodiode PD arranged sequentially in a lamination direction from anobject side to an imaging surface. Here, the microlens L is forincreasing light amount reaching the image pixels by concentratinglight, and effectively making a numerical aperture of the image pixelslarge. Also, regarding the color filter F, in a case, for example, of aprimary color Bayer array color filter, either of an R filter, G filteror B filter is provided in accordance with that pixel position.

In a case of the 2PD pixel structure shown in FIG. 3, two photodiodes PDare arranged in the imaging range of a single microlens L. Twophotodiodes PD are divided into left and right in the case where phasedifference in the horizontal direction is detected, and divided into topand bottom in the case where phase difference in the vertical directionis detected. As a result, a single pixel has two focus detection pixelsa and b.

On the other hand, in a case of the 4PD pixel structure shown in FIG. 3,four photodiodes PD are arranged in the imaging range of a singlemicrolens L. The four photodiodes PD are divided into four, namely left,right, top, and bottom, so that it is possible to detect phasedifference in the horizontal direction and in the vertical direction.Specifically, the four photo diodes PD are respectively arranged atupper left, lower left, upper right, and lower right positions. As aresult, a single pixel has four focus detection pixels a, b, c and d.

Also, as was described above, description will be given taking anexample where all pixels of the image sensor 208 have the 4PD pixelstructure. However, this does not mean that some pixels of the imagesensor 208 can be prevented from having a 4PD pixel structure or a 2PDpixel structure. In the event that all pixels of the image sensor 208have the 4PD pixel structure, a pixel signal output from each photodiodePD is a focus detection pixel signal.

Further, in a case where outputs of photodiodes PD are subjected tovertical two pixel addition using the circuit structure of FIG. 4, whichwill be described later, namely, in a case where (a+b) and (c+d) in FIG.3 are calculated, the output pixel signals constitute focus detectionpixel signals for detecting phase difference in the horizontal direction(vertical line detection). In a case where outputs of photodiodes PD aresimilarly subjected to horizontal two pixel addition, namely when (a+c)and (b+d) in FIG. 3 are calculated, pixel signals constitute focusdetection pixel signals for detecting phase difference in the verticaldirection (horizontal line detection).

In the case of the 4PD pixel structure shown in FIG. 3, of the focusdetection pixel signals for vertical line detection and the focusdetection pixel signals for horizontal line detection, one pairconstitute a pair of focus detection pixel signals for a firstpupil-division direction, and the other pair constitute a pair of focusdetection pixel signals for a second pupil-division direction. Further,in a case where outputs of photodiodes PD are similarly subjected tofour pixel addition, namely in the case where (a+b+c+d) in FIG. 3 iscalculated, this four pixel addition signal constitutes an image pixelsignal.

Next, a structural example of a pixel of the 4PD pixel structure will bedescribed using the circuit diagram shown in FIG. 4.

For a pixel of the 4PD pixel structure, as was shown in FIG. 3, fourphotodiodes PD1-PD4 are arranged at positions corresponding to a singlemicrolens L. Specifically, four photodiodes PD1-PD4 are respectivelyarranged at upper left, lower left, upper right, and lower rightpositions within a range of the microlens L in which an optical image isformed.

Transistors Tr1-Tr4 that function as switches are respectively connectedto the four photodiodes PD1-PD4. If control signals TX1-TX4 from thevertical scanning section 21 are respectively applied to the transistorsTr1-Tr4, on-off states of the transistors Tr1-Tr4 are respectivelycontrolled.

Each transistor Tr1-Tr4 is connected to a floating diffusion FD. Thismeans that if a transistor is turned on, signal charge of a photodiodePD corresponding to the transistor TR is transferred to the floatingdiffusion FD.

Also, one end of a transistor Tr5 that functions as a switch isconnected between each transistor Tr1-Tr4 and the floating diffusion FD,with the other end of the transistor Tr5 being connected to a powersupply voltage VDD. By applying a reset signal RES to transistor Tr5, onoff states for the power supply voltage VDD side and the floatingdiffusion FD side are controlled. With this structure, if the transistorTr5 is turned on, the floating diffusion FD is reset. Also, by turningthe transistor Tr5 on in a state where the transistors Tr1-Tr4 areturned on, the photodiodes PD1-PD4 are reset.

The floating diffusion FD is connected to an output terminal OUT via atransistor Tr6 that functions as a switch, and transistor Tr7 that isconnected to the power supply voltage VDD and functions as an amplifier.If a selection signal SEL is applied to the transistor Tr6, a voltagevalue of the floating diffusion FD is amplified by transistor Tr7, andread out from the output terminal OUT.

Next, operation for first readout (focus detection pixels priority mode)will be described using the timing charts shown in FIG. 5 and FIG. 6.

FIG. 5 is a timing chart showing an operational example of an imagesensor 208 when performing an electronic shutter operation at the timeof focus detection pixels priority mode. It should be noted that timesT1-T10 in FIG. 5, (and in FIG. 6-FIG. 11 which will be described later)represent times within the context of a single timing chart, and even ifthe same reference numerals (for example t1) that show times aredescribed in different timing charts, they do not represent the sametime. Also, the electronic shutter performs exposure time control forthe image sensor 208 as a result of transistors within the image sensor208 being controlled.

At time t2, if the reset signal RES is turned on (transistors Tr1-Tr6,that function as switches, are turned off apart from those that havebeen specifically stated as being on. The same applies in thefollowing.) the floating diffusion FD is reset. This reset signal RESremains on until the reset SIGNAL RES is turned off at time t4.

At time t3, if control signals TX1 and TX2 are turned on then since atthis point in time the reset signal RES is on signal charge of thephotodiodes PD1 and PD2 is reset.

After the reset signal RES has been turned off at time t4, if the resetsignal RES is turned on at time t7 the floating diffusion FD is resetagain. This reset signal RES remains on until the reset SIGNAL RES isturned off at time t9.

At time t8, if control signals TX3 and TX4 are turned on, then since atthis point in time the reset signal RES is on, signal charge of thephotodiodes PD3 and PD4 is reset.

Also, with this flow shown in FIG. 5, since there is no charge transferfrom the photodiodes PD1-PD4 to the floating diffusion FD, the floatingdiffusion FD holds a reset charge (labeled RES in the timing chart) fromtime t2 onwards.

FIG. 6 is a timing chart showing an operational example of an imagesensor 208 when performing pixel signal readout at the time of focusdetection pixels priority mode. FIG. 6 shows a readout operation of apixel signal performed after the exposure time (corresponding toso-called shutter speed) has elapsed from the electronic shutteroperation that was shown in FIG. 5.

At time t1, if the reset signal RES is turned on then the floatingdiffusion FD is reset. As a result of this reset, the floating diffusionFD holds a reset charge (RES). At time t2, if the select signal SEL isturned on, a voltage of the reset charge (RES) that has accumulated inthe floating diffusion FD is amplified by the transistor Tr7 and readout from the output terminal OUT.

At time t3, if the control signals TX1 and TX2 are turned on then signalcharge of photodiode PD1 (this signal charge is made PD1) and the signalcharge of photodiode PD2 (this signal charge is made PD2) aretransferred to the floating diffusion FD. As a result, the floatingdiffusion FD holds a charge (PD12+RES (it should be noted thatPD12=PD1+PD2)).

At time t4, if the select signal SEL is turned on, voltage of the charge(PD12+RES) that has accumulated in the floating diffusion FD is read outfrom the output terminal OUT, as described above. A reset voltage (resetnoise) corresponding to reset charge (RES) that is included in thisvoltage that has been read out at time t4 is removed, by a CDS circuitof the analog processing section 23, using a reset voltage that was readout at time t2, and it is made possible to obtain a signal voltagecorresponding to the charge (PD12). After that, although description isomitted, reset noise is similarly removed and only a signal component isdetected.

After that, at time t6, if the reset signal RES is turned on then thefloating diffusion FD is reset. The floating diffusion FD then holds areset charge (RES). At time t7, if the select signal SEL is turned on, avoltage of the reset charge (RES) that has accumulated in the floatingdiffusion FD is read out from the output terminal OUT.

At time t8, if the control signals TX3 and TX4 are turned on, thensignal charge of photodiode PD3 (this signal charge is made PD3) and thesignal charge of photodiode PD4 (this signal charge is made PD4) aretransferred to the floating diffusion FD. As a result, the floatingdiffusion FD holds a charge (PD34+RES (it should be noted thatPD34=PD3+PD4)).

At time t9, if the select signal SEL is turned on, voltage of the charge(PD34+RES) that has accumulated in the floating diffusion FD is read outfrom the output terminal OUT.

In this way, in the first readout, at time t4 an added value PD12, ofsignal charge output PD1 for the upper left photodiode and signal chargeoutput PD2 for the lower left photodiode, is read out. Also, at time t9an added value PD34, of signal charge output PD3 for the upper rightphotodiode and signal charge output PD4 for the lower right photodiode,is read out. Specifically, in the first readout, a pair of focusdetection pixel signals A and B of the image sensor 208 are respectivelyread out in readout 1.

Next, operation for second readout (image pixels priority mode) will bedescribed using the timing charts shown in FIG. 7 to FIG. 9.

FIG. 7 is a timing chart showing a first example of operation of animage sensor 208 when performing an electronic shutter operation at thetime of image pixels priority mode. It should be noted that in FIG. 7 toFIG. 11 points that are the same as in FIG. 5 and FIG. 6 will be omittedwhere appropriate, to simplify the following description.

At time t7, the reset signal RES is turned on, and the reset signal RESthat was turned on is turned off at the time t9. Then, at time t8 wherethe reset signal RES is on, the control signals TX1 to TX4 are turnedon, and signal charges for photodiodes PD1-PD4 are reset.

FIG. 8 is a timing chart showing a second example of operation of animage sensor 208 when performing an electronic shutter operation at thetime of image pixels priority mode. As will be described later withreference to FIG. 9, in pixel signal readout for image pixels prioritymode, signal charges for photodiodes PD1 and PD2 are read out at timet3, and signal charges for photodiodes PD1-PD4 are readout at time t8.Therefore, with the second example shown in FIG. 8, by performing resetof the photodiodes PD1-Pd4 for the electronic shutter operation twotimes, namely at time t3 and at time t8, variation in power supplyvoltage VDD becomes constant (steady).

Specifically, at time t2, the reset signal RES is turned on, and thereset signal RES that was turned on is turned off at time t4. Then, attime t3 where the reset signal RES is on, the control signals TX1 to TX4are turned on, and signal charges for photodiodes PD1-PD4 are reset.

Further, at time t7, the reset signal RES is turned on, and the resetsignal RES that was turned on is turned off at time t9. Then, at time t8where the reset signal RES is on, the control signals TX1 to TX4 areturned on, and signal charges for photodiodes PD1-PD4 are reset.

FIG. 9 is a timing chart showing an operational example of the imagesensor 208 when performing pixel signal readout at the time of imagepixels priority mode.

At time t1 the reset signal RES is turned on to reset the floatingdiffusion FD, and at time t2 a voltage of the reset charge (RES) is readout. Next, at time t3, signal charges for photodiodes PD1 and PD2 aretransferred to the floating diffusion FD. A voltage of the charge(PD12+RES) is then read outside time t4.

Next, at time t8, signal charges for photodiodes PD1 to PD4 aretransferred to the floating diffusion FD. At time t9 voltage of thecharge (PD1234+RES (it should be noted that PD1234=PD1+PD2+PD3+PD4)) isread out.

It should be noted that in the case of performing the operations shownin FIG. 9, exposure times are different for the charge read out at timet4 and the charge read out at time t9. However, since this difference inexposure time is small compared to actual exposure time (for example,1/30th of a second to 1/1000th of a second), it is considered that thereis almost no actual effect.

In this way, in the second readout, at time t4 an added value PD12, ofsignal charge output PD1 for the upper left photodiode and signal chargeoutput PD2 for the lower left photodiode, is read out. Also, at time t9an added value PD1234, of signal charge output PD1 for the upper leftphotodiode, signal charge output PD2 for the lower left photodiode,signal charge output PD3 for the upper right photodiode, and signalcharge output PD4 for the lower right photodiode, is read out.Specifically, with second readout, one of either of the pair of focusdetection pixel signals A and B (here, for example, it is made the focusdetection pixel signal A) of the image sensor 208, and an image pixelsignal (A+B) resulting from addition of the pair of focus detectionpixel signals A and B, are output.

Next, operation for third readout (image only mode) will be describedusing the timing charts shown in FIG. 10 and FIG. 11.

FIG. 10 is a timing chart showing an operational example of the imagesensor 208 when performing an electronic shutter operation at the timeof image only mode. At time t2, the reset signal RES is turned on, andthe reset signal RES that was turned on is turned off at time t4. Then,at time t3 where the reset signal RES is on, the control signals TX1 toTX4 are turned on, and signal charges for photodiodes PD1-PD4 are reset.

FIG. 11 is a timing chart showing an operational example of the imagesensor 208 when performing pixel signal readout at the time of imageonly mode. At time t1, the reset signal RES is turned on to reset thefloating diffusion FD. A voltage of the reset charge (RES) is then readout at time t2. At time t3, signal charges for photodiodes PD1 to PD4are transferred to the floating diffusion FD. A voltage of the charge(PD1234+RES) is read out at time t4. It is possible to detect a signalvoltage corresponding to charge (PD1234) by subtracting a voltage of thereset charge (RES) from a voltage of the charge (PD1234+RES) using theCDS circuit of the analog processing section 23.

With image only mode (third readout) performing the electronic shutteroperation as was shown in FIG. 10 and performing the pixel signal readout that was shown in FIG. 11, it is possible to acquire an image pixelsignal. However, it is not possible to acquire a focus detection pixelsignal. However, since readout is completed in one go, it is convenientat the time of display or storage of an image without performing focusdetection.

Next, operation of a modified example of first readout (focus detectionpixels priority mode) will be described using the timing charts shown inFIG. 19 and FIG. 20. The modified example of first readout is anoperation to read out only one focus detection pixel signal from thepair of focus detection pixel signals.

FIG. 19 is a timing chart showing an operational example of an imagesensor 208 when performing an electronic shutter operation at the timeof focus detection pixels priority mode. FIG. 19 is the same as theportions for t1 to t5 in FIG. 5. At time t2, if the reset signal RES isturned on then the floating diffusion FD is reset. This reset signal RESremains on until the reset SIGNAL RES is turned off at time t4. At timet3, if control signals TX1 and TX2 are turned on then since at thispoint in time the reset signal RES is on, signal charge of thephotodiodes PD1 and PD2 is reset.

FIG. 20 is a timing chart showing an operational example of an imagesensor 208 when performing one pixel signal readout of a pair of focusdetection pixel signals, in focus detection pixels priority mode. FIG.20 shows a readout operation of a pixel signal performed after theexposure time (corresponding to so-called shutter speed) has elapsedfrom the electronic shutter operation that was shown in FIG. 19. Itshould be noted that FIG. 20 shows the same operations as in t1 to t5 inFIG. 6.

At time t1 in FIG. 20, if the reset signal RES is turned on then thefloating diffusion FD is reset. As a result of this reset, the floatingdiffusion FD holds a reset charge (RES). At time t2, if the selectsignal SEL is turned on, a voltage of the reset charge (RES) that hasaccumulated in the floating diffusion FD is amplified by the transistorTr7 and read out from the output terminal OUT.

At time t3, if the control signals TX1 and TX2 are turned on then signalcharge of photodiode PD1 (this signal charge is made PD1) and the signalcharge of photodiode PD2 (this signal charge is made PD2) aretransferred to the floating diffusion FD. As a result, the floatingdiffusion FD holds a charge (PD12+RES (it should be noted thatPD12-PD1+PD2)).

At time t4, if the select signal SEL is turned on, voltage of the charge(PD12+RES) that has accumulated in the floating diffusion FD is read outfrom the output terminal OUT, as described above. A reset voltage (resetnoise) that is included in this voltage that has been read out at timet4 is removed, by a CDS circuit of the analog processing section 23,using a reset voltage that was read out at time t2. After that, althoughdescription is omitted, reset noise is similarly removed.

In this way, in the modified example of first readout, at time t4 anadded value PD12, of signal charge output PD1 for the upper leftphotodiode and signal charge output PD2 for the lower left photodiode,is read out. Specifically, with the modified example of first readout,of the pair of focus detection pixel signals A and B of the image sensor208, only focus detection pixel signal A is output. It should be notedthat instead of the added value PD12, added value PD34, for the signalcharge output PD3 of the upper right photodiode and signal charge outputPD4 of the lower right photodiode, may be read out.

Also, instead of the added value PD12, added value PD13, for the signalcharge output PD1 of the upper left photodiode and signal charge outputPD3 of the upper right photodiode, may be read out. In this case, addedvalue PD24 is calculated by subtracting added value PD13 from addedvalue PD1234 resulting from third readout of a still picture frame, andit becomes possible to perform correlation calculation for differentpupil-division directions based on the added values PD13 and PD24.

By using the modified example of focus detection pixel priority mode(first readout) where an electronic shutter operation such as was shownin FIG. 19 is performed and pixel signal read out such as was shown inFIG. 20 is performed, and image pixel priority mode (second readout)where an electronic shutter operation such as was shown in FIG. 7 orFIG. 8 is performed and image signal readout such as was shown in FIG. 9is performed, then as will be described later, by subtracting one sideopening pixel data (for example PD12) that has been read out, from anadded value for left and right opening pixels (PD1234), other sideopening pixel data (PD34) is obtained, and it is possible to acquireboth an image pixel signal and a focus detection pixel signal. In thiscase, however, readout is performed twice.

Next, operation of the imaging device 1 of this embodiment will bedescribed using the flowcharts shown in FIG. 12A and FIG. 12B. Here,FIG. 12A and FIG. 12B show operation at the time an AF mode of theimaging device 1 is continuous AF mode. Continuous AF mode is an AF modethat is suitable for a moving subject, and is an AF mode where focusingis performed continuously so as to track a subject.

If an on operation of the power supply of the imaging device 1 by theuser is detected, the flow for camera power supply on shown in FIG. 12Ais commenced. If the power supply on operation is detected, it is firstdetermined whether or not a 1st release is on (S1). Here, the CPU 212determines whether or not a 1st release switch of a release buttonwithin the operation section 206 is in an on state. The 1st releaseswitch is a switch that is put in an on state in response to a halfpress operation of a release button by the user, for example. If theuser focuses on a subject and decides on exposure, the release button ispressed down halfway.

If the result of determination in step S1 is that the 1st release switchis not on, acquisition of live view (LV) is performed (S3). Here, theCPU 212 outputs a control signal to the drive section 204 so as to putthe mechanical shutter 202 in a fully-open state, as well as outputtinga control signal to the lens CPU 106 so as to move the aperture 102 b bya given amount (for example, open aperture wider). After that, the CPU212 outputs a control signal to the image sensor 208 and imaging for LVdisplay is commenced by the image sensor 208 Every time imaging for LVdisplay is completed, the element control section 29 commences readoutof pixel signals from the pixel section 22. It should be noted that atthe time of pixel signal readout, readout is performed using thirdreadout described above, and the element control section 29 adds pixelsignals of the same opening (color) output from the pixel section 22.Pixel data for display that has been output from the image sensor 208 isstored in the DRAM 226.

Also, the CPU 212 performs live view (LV) display in step S3. At thistime, the CPU 212 causes generation of display image data in the imageprocessing circuit 214. In response to this, the image processingcircuit 214 performs necessary processing on the pixel data for displayto generate display image data for display. The CPU 212 displays an LVimage on the display section 222 based on display image data that hasbeen generated by the image processing circuit 214. Once the LV displayhas been performed, processing returns to step S1.

If the result of determination in step S1 is that the 1st release switchis on, exposure and readout for AF and LV are performed (S5). Here, theCPU 212 performs imaging and readout for autofocus (AF) and live view(LV) display. Imaging and readout for AF are performed using the abovedescribed second readout. With second readout, one of either of the pairof focus detection pixel signals A and B (here, for example, it is madethe focus detection pixel signal A) from the image sensor 208, and animage pixel signal (A+B) resulting from addition of the pair of focusdetection pixel signals A and B, are output. By subtracting the focusdetection pixel signal A from the added image pixel signal (A+B) it ispossible to obtain the focus detection pixel signal B. The added imagepixel signal (A+B) is then used for LV display. Alternatively, imagingand readout for AF may be performed using the above described firstreadout, and imaging and readout for LV display may be performed usingthe above described third readout, and the first readout and the thirdreadout may be alternately and repeatedly executed.

Focus detection using a phase difference method Is performed using thefocus detection pixel signals A and B that were calculated in step S5.In this step, data of the focus detection pixel signals A and B for AFis stored in the DRAM 226. It should be noted that in a case where the4PD pixel structure is adopted, it is possible to calculate a pair offocus detection pixel signals respectively for the horizontal direction(left and right direction) and the vertical direction (up and downdirection), and it is possible to calculate the phase difference for therespective directions. Also, the image pixel signal (A+B) is stored inthe DRAM 226 as display pixel data for LV. Here, live view display isbased on the pixel data for display that has been stored in the DRAM226.

Next, correlation calculation and reliability determination areperformed (S7). Here, the CPU 212 executes focus detection calculationusing the focus detection circuit 218. The focus detection circuit 218performs correlation calculation using focus detection pixel data (focusdetection pixel signals A and B) that constitute a pair, among focusdetection pixel data that is stored in the DRAM 226. Focus detectionpixel data that constitutes a pair in the case of phase differencedetection in the horizontal direction is the left opening pixel data 1and the right opening pixel data r, while the focus detection pixel datathat constitutes a pair in the case of phase difference detection in thevertical direction is the top opening pixel data t and the bottomopening pixel data b. After correlation calculation, the focus detectioncircuit 218 performs reliability detection for focus detection.Reliability determination is determination based on contrast obtainedfrom pixel data, for example, and/or correlation values etc. calculatedas results of correlation calculation.

Once correlation calculation and reliability determination have beenperformed, next focus deviation amount is detected (S9). Here, the focusdetection circuit 218 performs focus deviation amount calculation.Specifically, the focus detection circuit 218 calculates focus deviationamount for an in-focus position of the focus lens 102 a from a two-imageinterval value for focus detection regions in which it has beendetermined that the result of reliability determination in step S7 ishigh reliability (image shift amount corresponding to extreme value ofcorrelation value).

If a focus deviation amount has been detected, next an area is selected(S11). Here, the focus detection circuit 218 performs area selectionprocessing in order to select a focus detection region in which tocalculate target focus lens position used in drive of the focus lens 102a. Area selection processing selects a focus detection region exhibitinga focus deviation amount for which a target focus lens position has beencalculated in accordance with the closest subject distance (that is, atthe closest range). Also, the area selection processing is not limitedto the closest range, and it is also possible to select an area in whicha person's face exists, and it is also possible have an area that hasbeen selected manually by the user. It should be noted that areaselection may also be performed by the CPU 212.

If area selection has been performed, it is next determined whether ornot there is a focused state (S13). Here, the CPU 212 determines whetheror not the focus lens 102 a is in a focused state. This determination isdetermination as to whether or not a focus deviation amount for thefocus detection region that was selected in the area selectionprocessing, for example, is within a previously determined permissiblerange. When the focus deviation amount is within a permissible range afocused state is determined.

If the result of determination in step S13 is not a focused state, thefocus lens is driven (S15). Here, the CPU 212 outputs a control signalto the lens CPU 106 so as to drive the focus lens 102 a to a targetfocus lens position that was calculated for the focus detection regionthat was selected in step S11. The lens CPU 106 receives this controlsignal and drives the focus lens 102 a by means of the drive section104. Once the focus lens 102 a has been driven processing returns tostep S1.

If the result of determination in step S13 is a focused state, exposureand readout for AF and LV are performed (S17). Here, the CPU 212performs the same imaging and readout for autofocus (AF) and live view(LV) display as was performed in step S5. As was described previously, apixel signal is read out from the image sensor 208 using second readout,focus detection pixel data for AF is stored in the DRAM 226, and displaypixel data for LV is stored in the DRAM 226. Also, live view (LV)display is performed on the display section 222 using the display pixeldata for LV.

Next, correlation calculation and reliability determination areperformed (S19). Here, similarly to step S7, the CPU 212 causesexecution of focus detection calculation by the focus detection circuit218, using a pixel signal that was read out in step S17. The focusdetection circuit 218 performs correlation calculation using focusdetection pixel data that constitute a pair, among focus detection pixeldata that is stored in the DRAM 226. After correlation calculation, thefocus detection circuit 218 performs reliability detection for focusdetection. If correlation calculation and reliability determination havebeen performed, next, similarly to step S9, focus deviation amount isdetected (S21), and, similarly to step S11, area selection is performed(S23).

Once area selection has been performed, next history information issaved (S25). Here, the focus detection circuit 218 saves informationrelating to focus detection as history information in the DRAM 226, forexample. Information relating to focus detection includes, for example,information on the focus deviation amount that was calculated in stepS21, and information on the focus detection region that was selectedinstead S23. It should be noted that saving of history information mayalso be the CPU 212 saving information relating to focus detection inthe DRAM 226.

Once history information has been saved, it is next determined whetheror not the 2nd release switch is on (S31). Here, the CPU 212 determineswhether or not the 2nd release switch within the operation section 206has been turned on. The 2nd release switch is a switch that is put in anon state in response to a full press operation of a release button bythe user, for example. The user presses the release button down fully inthe case of shooting a still image.

If the result of determination in step S31 is that the 2nd releaseswitch is not on, it is next determined whether or not there is afocused state (S33). Here, the CPU 212 determines whether or not thefocus lens 102 a is in a focused state, Similarly to step S13. If theresult of this determination is a focused state, processing returns tostep S17.

If the result of determination in step S33 is not a focused state, thefocus lens is driven (S35). Here, similarly to step S15, the CPU 212moves the focus lens 102 a to an appropriate focus lens position basedon the focus deviation amount. If focus lens drive has been performed,processing returns to step S17.

If the result of determination in step S31 is that the 2nd releaseswitch is on, next moving body estimation computation is performed(S37). Here, the CPU 212 causes execution of moving body estimationcomputation by the focus detection circuit 218. As was describedpreviously the flow shown in FIG. 12A and FIG. 12B is a case wherecontinuous shooting mode has been set, and in a case where a subject ismoving also, focus control is performed to estimate subject movement soas to achieve focus for each frame that is continuously shot. Movingbody estimation computation is estimating a position the focus lens 102a should be driven to for focusing on a subject for the current stillpicture exposure time, based on history of results of previous focusdeviation amount calculation (focus lens position) that were stored instep S25.

If moving body estimation computation has been performed, next a shutteroperation is commenced (S39). Here, the CPU 212 causes commencement ofoperation of the mechanical shutter 202 in order to perform imaging(actual exposure) for still picture acquisition. This operation of themechanical shutter 202 includes opening and closing operations of themechanical shutter 202 before and after actual exposure, and a fullyopen operation of the mechanical shutter 202 after actual exposure, inorder to commence imaging for live view and AF. The CPU 212 firstswitches control signals of the drive section 204 so as to put themechanical shutter 202 in a fully closed state. Then, after actualexposure has been performed in step S43, the CPU 212 controls the drivesection 204 soul is to put the mechanical shutter 202 in a fully openstate.

If the shutter operation has been commenced, the aperture and lens driveare simultaneously commenced (S41). Here, the CPU 212 instructs the lensCPU 106 so as to drive the focus lens 102 a and the aperture 102 b atthe same time, and operations for both components are commenced. Here,drive position for the focus lens 102 a is a position that was estimatedin the moving body estimation computation of step S37. Also, openingamount of the aperture 102 b is an opening amount corresponding to theaperture value that has been calculated based on subject brightness thatwas estimated as a result of previous photometry computation.

Next, actual exposure, and AF exposure and readout, are performed (S43).Here, the CPU 212 causes actual exposure to start, and after actualexposure has been completed reads out pixel signals from the imagesensor 208 using the previously described third readout. Actual exposureis imaging in order to acquire image data for storage. With actualexposure, the CPU 212 causes commencement of imaging of the image sensor208. After an exposure period is complete, the CPU 212 reads out pixelsignals from each light receiving section of the image sensor 208 asstill picture pixel signals. After readout of the still picture pixelsignals, the CPU 212 causes processing for generating an image pixelsignal for storage to be performed in the image processing circuit 214.In response to this the image processing circuit 214 generates stillpicture data for storage by performing necessary processing to generateimage data full storage. After completion of image processing, the CPU212 compresses the still picture data storage using the imagecompression and expansion section 216. After completion of compression,the CPU 212 stores the still picture data for storage that has beencompressed in the storage medium 230 as an image file.

Also, in step S43, while the 2nd release switch is on, the CPU 212continues to alternate between actual exposure and readout (stillpicture), and AF exposure and readout (phase difference detection) as inFIG. 13, which will be described later (in the case of Yes in step S57,which will be described later, processing returns to step S43). AFexposure is performing exposure with exposure control values (forexample, electronic shutter speed and ISO sensitivity) at which focusdetection pixels will achieve appropriate exposure. Also, AF exposure isperformed between one actual exposure and another actual exposure. Thispixel signal readout for AF exposure is performed with first readout toread out a signal from the focus detection pixels. Timing for actualexposure and AF exposure will be described later using FIG. 13.

Once the AF exposure and readout of step S43 have been performed, nextcorrelation calculation and reliability determination are performed(S45). After actual exposure and readout have been performed in stepS43, AF exposure and readout are performed, and processing from stepsS45 to S53 is performed. The correlation calculation and reliabilitydetermination of step S45 is performed similarly to steps S7 and S19,using a signal from focus detection pixels that was acquired in stepS43.

Once correlation calculation and reliability determination have beenperformed, next focus deviation amount is detected (S47). Here, focusdeviation amount is detected similarly to steps S9 and S21, based on thecorrelation calculation that was calculated in step S45. Next, areaselection is performed (S49). Here, the focus detection circuit 218performs area selection, similarly to steps S11 and S23, based on afocus deviation amount that was detected in step S47.

Once area selection has been performed, next history information issaved (S51). Before continuous shooting of still pictures and duringcontinuous shooting of still pictures, AF exposure is performedcontinuously (refer to S17 and S43), and focus deviation amount isdetected. The focus detection circuit 218 saves information relating tofocus detection as history information in the DRAM 226, for example.

Once history information has been saved, moving body estimationcomputation is performed (S53). Here, similarly to step S37, moving bodyestimation computation is performed based on history information thatwas saved in step S53. If moving body estimation computation has beenperformed, next, focus lens drive is performed (S55). Here, similarly tostep S37, position of the focus lens 102 a at the time of actualexposure is estimated, and the focus lens 102 a is moved to thisposition.

It is next determined whether or not the 2nd release switch is on (S57).If the user is continuing rapid shooting, the release button will stillbe pressed down fully. In this step, the CPU 212 determines whether ornot the 2nd release switch within the operation section 206 has beenturned on. If the result of this determination is that the 2nd releaseswitch is off, step S43 is returned to.

On the other hand if the result of determination in step S57 is that the2nd release switch is not on, it is next determined whether or not the1st release switch is on (S59). Here, similarly to step S1, the CPU 212determines whether or not a 1st release switch of a release buttonwithin the operation section 206 is in an on state. If the result ofthis determination is that the 1st release switch is on, processingreturns to step S17.

On the other hand, if the result of determination in step S59 is thatthe 1st release switch is not on, it is determined whether or not thecamera power supply is off (S61). Here, the CPU 212 determines whetheror not to turn the power supply of the camera body 200 off. For example,if the user operates the operation section 206 and instructs powersupply off, or if the user has not operated the operation section 206for a specified time, it is determined to turn the power supply off. Ifthe result of this determination is not to turn the power supply of thecamera body 200 off, processing returns to step S1. On the other hand ifit has been determined in step S61 to turn the power supply of thecamera body 200 off, the processing shown in FIG. 12A and FIG. 12B isterminated.

In this way, in the flow of FIG. 12A and FIG. 12B, in a case where therelease button has been pressed down fully actual exposure (stillpicture frame) and AF exposure (phase difference frame) are alternatelyand repeatedly executed (refer to S43). In a case where AF exposure hasbeen performed, correlation calculation and detection of focus deviationamount are performed based on a pixel signal of focus detection pixelsthat has been acquired as a result of this AF exposure, and moving bodyestimation and focus lens drive are performed based on this detectionresult.

Next, the actual exposure (still picture frame) and AF exposure (basedifference frame) in step S43 will be described using FIG. 13. In FIG.13, VD represents timing of a vertical synchronization signal, and T1 toT11 in the horizontal axis direction represent times. Also, the verticalaxis direction represents exposure timing for each line of the pixelsection 22 of the image sensor. Left and right edges of parallelogramsrepresenting each frame represent exposure commencement time andexposure completion time for each respective line. At time T1 initial(first frame) actual exposure is commenced, and exposure is commencedsequentially from the first line of the pixel section 22 of the imagesensor 208, at time T2 exposure for the first line is completed, andafter that actual exposure for the second frame commences at time T4,actual exposure for the third frame commences at time T7, and actualexposure for the fourth frame commences at time T10. For respectiveactual exposures, as image data for a still picture, image data for animage only is acquired using the previously described third readout.

Also, between one actual exposure and another actual exposure, pixeldata of focus detection pixels for phase difference detection are readout. B Specifically, at time T3 exposure for initial (first frame) phasedifference detection is commenced, exposure for the first line iscompleted at time T4, and after that exposure for phase differencedetection of the second frame commences at time T6, and exposure forphase difference detection of the third frame commences at time T9. Inthe respective exposure for phase difference detection, as image datafor phase difference detection, image data for phase differencedetection is acquired using the previously described first readout. Itshould be noted that at the time of phase difference detection, imagedata for phase difference detection corresponding to openings of oneside is acquired by the previously described second readout, and phasedifference detection image data corresponding to openings of the otherside may be obtained by subtracting image data for phase differencedetection from image data that has been acquired for a still picture. Itshould be noted that in FIG. 13, the reason that change in times forcommencement and completions of exposure for each line for phasedifference detection is different to that for a still picture is thatwhile all image pixels are read out for a still picture, for phasedifference detection pixels addition and thinning are performed, as willbe described later.

In this way, with this embodiment, at the time of acquiring image dataof a still picture with rapid shooting, exposure for phase differencedetection (phase difference frame) is performed between one actualexposure for still picture shooting (still picture frame) and anotheractual exposure. Specifically, imaging operations to generate pixelsignals for storage, and imaging operations to generate phase differencedetection pixel signals, are performed alternately. This means that itis possible to read out a pixel signal for storage, and to read out aphase difference detection pixel signal, at high speed without theoccurrence of invalid frames. Accordingly, since an image for storage isnot degraded, and it is possible to perform focus detection of thesubject, it is possible to acquire a focused image even with a movingsubject when high-speed rapid shooting is in progress. Also, the imagingdrive mode for alternately and repeatedly executing a shooting frame(third readout) and a phase difference frame (first readout) may be setin advance within the image sensor 208. As a result, it is possible toreduce the amount of communication from the CPU 212 to the image sensor208 at the time of rapid shooting of still pictures, such as commandsfor switching imaging drive mode. It is also possible to avoid theoccurrence of invalid frames accompanying switching of imaging drivemode within the image sensor 208 at the time of continuous shooting, andto avoid time lag caused by such invalid frames.

Next, an example of pixel addition at the time of first readout will bedescribed using FIG. 14. With the example shown in FIG. 14, setting isperformed so as to perform only addition in the vertical direction,which is the second pupil division direction, without performingaddition for the horizontal direction, which is the first pupil divisiondirection. Specifically, in the horizontal direction pixel signalsetting is performed so that for the same openings (associated leftopenings or associated right openings) are subjected to 1/1 pixeladdition (that is, no addition), and in the vertical direction pixelsetting is performed so that signals for the same openings (associatedleft openings or associated right openings) are subjected to 5/9 pixeladdition. Here, 5/9 pixel addition means adding of 5 pixels R1corresponding to m1 to n9, or the addition of 5 pixels Gbl correspondingto n2 to n10, among 9 pixels in the vertical direction (with the exampleof m1 in FIG. 14, 9 respective pixels for R1 corresponding to n1 to n17,or 9 respective pixels Gbl corresponding to n2 to n18). A number ofadditions of pixel signals may be set appropriately in accordance withframe rate, for example.

FIG. 15 shows added pixels for phase difference pixels (focus detectionpixels) of a phase difference frame, for every column. The example hereshows a case where only G pixels are extracted, in order to increasespeed. For example, with column m1 an added value Gr_L for a line ofleft opening Grl pixels n1, n3, n5, n7 and n9 of column m3 in FIG. 14 iscalculated, and with column m2 an added value Gr_R for a line of rightopening Grr pixels n1, n3, n5, n7 and n9 of column m4 in FIG. 14 iscalculated. Also, similarly, with column m1 an added value Gb_L for aline of left opening Gbl pixels n2, n4, n6, n8 and n10 of column m1 inFIG. 14 is calculated, and with column m2 an added value Gb_R for a lineof right opening Gbr pixels n2, n4, n6, n8 and n10 of column m2 in FIG.14 is calculated. Other addition pixel values in FIG. 15 are alsocalculated by means of the calculation processing with the sameprocedure.

In this way, for each line, addition values Gr_L, Gr_R, Gb_L and Gb_Rfor 5 pixels of Grl and Grr pixels, or Gbl and Gbr pixels, arecalculated, and stored in memory in the format shown in FIG. 15. Thecorrelation calculation of step S45 etc. is implemented using theseaddition values Gr_L and Gr_R, and Gb_L and Gb_R. It should be notedthat the description in FIG. 15 can be applied to up to column m40 thatis not shown in FIG. 14, which is only showing up to column m20. Forexample, R pixels Gr of lines n1 to n9 in column 20 of FIG. 15 are addedvalues Gr_R for right opening Grr pixels of lines n1, n3, n5, n7 and n9of column m40 that is not shown in FIG. 14.

By performing 5/9 pixel addition such as shown in FIG. 14 and FIG. 15,the number of lines of pixel signals is compressed. By reducing thenumber of lines of pixel signals, readout time for the pixel signals isshortened. On the other hand, number of columns of pixel signals doesnot change, which means that it is possible to ensure detectionprecision for phase difference in the horizontal direction. A focusdetection pixel signal generated based on a pixel signal has informationon phase difference in a substantially diagonal direction. It should benoted that with the example of FIG. 14, although setting has beenperformed so as to add five pixel signals for the vertical direction,the number of additions in the vertical direction may be five pixels orless.

After the setting of pixel readout, such as shown in FIG. 14 and FIG.15, the CPU 212 outputs control signals to the image sensor 208 so as toperform imaging at an exposure time necessary to generate a focusdetection pixel signal. This exposure time is set based on subjectbrightness etc. Also, receiving input of control signals from the CPU212, the element control section 29 commences imaging for each line ofthe pixel section 22. The element control section 29 then controls avertical scanning circuit 304 to sequentially output pixel signals fromlines of the pixel section 22 for which imaging has been completed.

Next, another example of readout of phase difference pixels in a casewhere still picture frames and phase difference frames are alternatelyand repeatedly executed will be described using FIG. 16 and FIG. 17.This example corresponds to the previously described modified example offirst readout. With this example, pixel data of either one of leftopening pixels or right opening pixels is read out, as shown in themodified example of first readout in FIG. 19 and FIG. 20. Then, bysubtracting pixel data for one side opening that has been read out (forexample, PD12) from an added value (PD1234) for left and right openingpixels being acquired in response to a still picture frame, the otherside opening pixel data (PD34) is obtained. Specifically, in step S43 ofFIG. 12B, the CPU 212 obtains a right opening pixels value lm_difr(PD34) by subtracting a left opening pixels value lm_difl (PD12) from anall pixels value lm_ful (PD1234) for R pixels, GR pixels, Gb pixels andB pixels, shown in FIG. 16. As a result, it is possible to calculateleft opening pixel values (PD12) and right opening pixel values (PD34)among the focus detection pixels, and it is possible to performcorrelation calculation in step S45 using this focus detection pixelvalue that has been calculated.

FIG. 17 shows added pixels for phase difference pixels (focus detectionpixels) of a phase difference frame, for every column. It should benoted that in FIG. 17 also, addition processing for phase differencepixels (focus detection pixels) of a phase difference frame for everycolumn is performed in the same way as shown in FIG. 15.

When reading out the phase difference frame shown in FIG. 16, only pixelvalues (PD12) of left opening pixels are read out. This means that form1, m3, m5, m9, m11, m15, m17 and m19, which are left opening pixelcolumns, a value for which addition processing has been performed isread out, while addition processing is not performed for pixel valuessuch as m2, m4, m6 and m8, which are right opening pixel columns, andthere is no readout. With the column m1 in FIG. 17, Grl pixel values forlines n1, n3, n5, n7 and n9 of column m3 in FIG. 14 are added and madean added value Gr_L, while Gbl pixel values for lines n2, n4, n6, n8 andn10 of column m1 in FIG. 14 are added and made an added value Gb_L, andthese values are stored in memory. For other lines also, addition isperformed using the same procedure and added values are stored inmemory.

Compared to the case of first readout that was described using FIG. 14and FIG. 15, with the modified example of first readout, since it isonly necessary to read out one side opening pixel column, it is possibleto make the system high speed. However, for other side opening pixelcolumns pixel values are obtained by computational processing by the CPU212 from pixel values that have been acquired at different exposuretimings, which means that detection precision is slightly lowered. Thismeans that when focus detection precision is required to be high, forexample, in a case of extremely high precision and at the time ofwide-open aperture etc., the modified example of first readout should beadopted. In the case of super high speed still picture rapid shooting,if a time difference between a still picture frame and a phasedifference frame is extremely small, this modified example should beused.

Next, an initial valid frame, at the time of switching to imaging drivemode, when actual exposure and pressing down of the 1st release havebeen performed, will be described using FIG. 18A and FIG. 18B. It shouldbe noted that in FIG. 18A and FIG. 18B, the horizontal axis representstime.

FIG. 18A is a drawing showing switching of an imaging drive mode,specifically from a state where live view display is being performedbefore actual exposure, to actual exposure. In a state where live viewdisplay is being performed (T21) without the user operating the releasebutton, if time T21 a is reached the user presses the release buttondown fully and the 2nd release switch is turned on. The CPU 212 detectsthe 2nd release switch being turned on.

At time T22, the image sensor 208 performs readout of pixel values forlive view display. Readout of pixel values is also being performed attime T23, but in response to the 2nd release switch being on the CPU 212issues an imaging drive mode switching command to the image sensor 208at this time. Specifically, the CPU 212 issues a command to the imagesensor 208 to switch from pixel signal for live view display generatingmode to an imaging drive mode in which pixel signal generation for astill picture frame and pixel signal generation for phase differencedetection are alternately executed. The element control section 29 ofthe image sensor 208 receives this switching command via the inputsection 28. Then, the element control section 29 controls each blockwithin the image sensor 208 to switch from pixel signal for live viewdisplay generating mode to an imaging drive mode in which pixel signalgeneration for a still picture frame and pixel signal generation forphase difference detection are alternately executed. At that time, inthe imaging drive mode in which pixel signal generation for a stillpicture frame and pixel signal generation for phase difference detectionare alternately executed, the CPU 212 issues a command to the imagesensor 208 so as to execute an initial pixel signal for still pictureframe generation.

At time T23 in FIG. 18A, there is a switch from pixel value for liveview display readout to pixel value for still picture frame readout. Asa result of this, pixel values at time T23 are treated as pixel valuesof an invalid frame. Then, readout is alternately performed for pixelvalues of a still picture frame at time T24, and pixel values for aphase difference frame time T25. After that also, while the 2nd releaseis on, pixel values for still picture frames and pixel values for phasedifference frames are alternately output from the image sensor 208, andare respectively read out and processed by the image processing circuit214 and the focus detection circuit 218.

In this way, it is ensured that a still picture frame is handled as aninitial valid frame after the 2nd release has been pressed down. As aresult, since the initial valid frame constitutes a still pictureexposure, it is possible to make shutter release time lag as short aspossible.

FIG. 18B is a drawing showing switching of imaging drive mode from thestate where live view display is being performed, when the 1st releaseis pressed down and there is a switch to AF exposure. In a state wherelive view display is being performed (T31) without the user operatingthe release button, if time T31 a is reached the user presses therelease button down halfway and the 1st release switch is turned on. TheCPU 212 detects the 1st release being turned on.

At time T32, the image sensor 208 performs readout of pixel values forlive view display. Readout of pixel values is also being performed attime T33, but in response to the 1st release switch being on the CPU 212issues an imaging drive mode switching command to the image sensor 208at this time. Specifically, the CPU 212 issues a command to the imagesensor 208 to switch from pixel signal for live view display generatingmode to an imaging drive mode in which pixel signal generation for phasedifference detection and generation of a pixel signal for live view (fordisplay) are alternately executed. The element control section 29 of theimage sensor 208 receives this switching command via the input section28. Then, the element control section 29 controls each block within theimage sensor 208 to switch from an imaging drive mode in which a pixelsignal for live view display is generated to an imaging drive mode inwhich pixel signal generation for phase difference detection andgeneration of a pixel signal for live view (for display) are alternatelyexecuted. At that time, in the imaging drive mode in which pixel signalgeneration for phase difference detection and generation of a pixelsignal for live view (for display) are alternately executed, the CPU 212issues a command to the image sensor 208 so as to execute an initialpixel signal for still phase difference detection generation operation.

At time T23 in FIG. 18A, there is a switch from pixel value for liveview display readout to pixel value for phase difference detectionreadout. As a result of this, pixel values at time T33 are treated aspixel values of an invalid frame. Then, readout is alternately performedfor pixel values of a phase difference frame at time T34, and pixelvalues for a display frame time T35. After that also, while the 1strelease is on, pixel values for phase difference frames and pixel valuesfor display frames are alternately output from the image sensor 208, andare respectively read out and processed by the focus detection circuit218 and the image processing circuit 214.

In this way, it is ensured that an initial valid frame after the 1strelease has been pressed down constitutes a frame for phase differencedetection. As a result, since the initial valid frame constitutes aphase difference detection frame, it is possible to make the time tocommencement of AF as short as possible.

As has been described above, with the one embodiment of the presentinvention, pixel signals are alternately read out from the image sensorduring continuous AF with rapid shooting. With this alternate read out,readout for still picture exposure and readout from photodiodes thathave been divided for focus detection (PD divided exposure) areperformed alternately. Specifically, a first imaging operation, in whichpixel signals corresponding to pupil division direction are combined anda pixel signal for storage is generated, and a second imaging operationin which pixel signal corresponding to a pupil division direction aregenerated, are alternately performed. This means that there is nodegradation in a still image, and moreover, is also possible to performaccurate focus detection.

Also, with the one embodiment of the present invention, readout fromphotodiodes that have been divided for focus detection (PD dividedexposure) involves reading out of both photodiodes for one side openings(for example, left openings, top openings) and photodiodes for the otherside openings (for example, right openings, bottom openings) (refer, forexample, to FIG. 5 and FIG. 6, and first readout in FIG. 14 and FIG.15). This means that it is possible to obtain accurate output ofrespective photodiodes for one side opening on the other side opening,and it is possible to increase focus detection precision.

Also, with the one embodiment of the present invention, readout fromphotodiodes that have been divided for focus detection (PD dividedexposure) involves reading out of either one of photodiodes for one sideopenings (for example, left openings, top openings) or photodiodes forthe other side openings (for example, right openings, bottom openings)(refer, for example, to FIG. 7 to FIG. 9, FIG. 14, and second readout inFIG. 16 and FIG. 17). Since only either one of the photodiodes is readout, it is possible to expect shortening of readout time.

Also, with the one embodiment of the present invention, at the time of2nd release there is a switch to an imaging mode where pixel signals arealternately read out from the image sensor. For an initial frame at thetime of switching an imaging operation is performed for still pictureexposure. As a result it is possible to shorten time lag until stillpicture shooting.

It should be noted that with the one embodiment of the presentinvention, some or all of the focus detection circuit 218, imageprocessing circuit 214, image compression and expansion section 216,exposure control circuit 220 etc. may be integrated with the CPU 212 andthe peripheral circuitry of the CPU. It is also possible for the focusdetection circuit 218, image processing circuit 214, image compressionand expansion section 216, exposure control circuit 220 etc. to have ahardware structure such as gate circuits that have been generated basedon a programming language that is described using Verilog, and also touse a hardware structure that utilizes software such as a DSP (digitalsignal processor). Suitable combinations of these approaches may also beused.

Also, with this embodiment, an instrument for taking pictures has beendescribed using a digital camera, but as a camera it is also possible touse a digital single lens reflex camera or a compact digital camera, ora camera for movie use such as a video camera, and further to have acamera that is incorporated into a mobile phone, a smartphone, a mobileinformation terminal, personal computer (PC), tablet type computer, gameconsole etc., a medical camera, or a camera for a scientific instrumentsuch as a microscope, a camera for mounting on a vehicle, a surveillancecamera etc. In any event, it is possible to apply the present inventionas long as a device is for shooting that, when performing exposure forstorage, also performs exposure for focus detection.

Also, among the technology that has been described in thisspecification, with respect to control that has been described mainlyusing flowcharts, there are many instances where setting is possibleusing programs, and such programs may be held in a storage medium orstorage section. The manner of storing the programs in the storagemedium or storage section may be to store at the time of manufacture, orby using a distributed storage medium, or they be downloaded via theInternet.

Also, with the one embodiment of the present invention, operation ofthis embodiment was described using flowcharts, but procedures and ordermay be changed, some steps may be omitted, steps may be added, andfurther the specific processing content within each step may be altered.It is also possible to suitably combine structural elements fromdifferent embodiments.

Also, regarding the operation flow in the patent claims, thespecification and the drawings, for the sake of convenience descriptionhas been given using words representing sequence, such as “first” and“next”, but at places where it is not particularly described, this doesnot mean that implementation must be in this order.

As understood by those having ordinary skill in the art, as used in thisapplication, ‘section,’ ‘unit,’ ‘component,’ ‘element,’ ‘module,’‘device,’ ‘member,’ ‘mechanism,’ ‘apparatus,’ ‘machine,’ or ‘system’ maybe implemented as circuitry, such as integrated circuits, applicationspecific circuits (“ASICs”), field programmable logic arrays (“FPLAs”),etc., and/or software implemented on a processor, such as amicroprocessor.

The present invention is not limited to these embodiments, andstructural elements may be modified in actual implementation within thescope of the gist of the embodiments. It is also possible form variousinventions by suitably combining the plurality structural elementsdisclosed in the above described embodiments. For example, it ispossible to omit some of the structural elements shown in theembodiments. It is also possible to suitably combine structural elementsfrom different embodiments.

What is claimed is:
 1. An image sensor, comprising: a plurality ofphoto-diodes arranged divided in a specified pupil division direction,so that a pixel signal is generated by subjecting respective light flux,that passes through different exit pupil regions of an imaging opticalsystems for a single micro-lens, to photoelectric conversion; and acontrol circuit that implements an imaging mode for alternately andrepeatedly executing a first imaging operation and a second imagingoperation; wherein the first imaging operation combines pixel signalscorresponding to the pupil division direction and generates and outputsa pixel signal for storage, and the second imaging operation generatesand outputs a pixel signal corresponding to the pupil divisiondirection, for focus detection.
 2. The image sensor of claim 1, wherein:the second imaging operation only outputs a pixel signal correspondingto pupil division directions of some among the plurality ofphoto-diodes.
 3. The image sensor of claim 1, wherein: the secondimaging operation outputs pixel signals corresponding to pupil divisiondirections of all of the plurality of photo-diodes.
 4. The image sensorof claim 1, wherein: the image sensor has a plurality of imaging modes,and in a case where an imaging mode, among the plurality of imagingmodes, has been set whereby the first imaging operation and the secondimaging operation are alternately and repeatedly executed, the firstimaging operation is executed before the second imaging operation.
 5. Afocus detection device having the image sensor of claim 1, comprising:an image processing circuit that, when the image sensor is executing thefirst imaging operation, performs processing to store still image databased on pixel signals output from the image sensor; a focus detectioncircuit that, when the image sensor is executing the second imagingoperation, performs processing for focus detection based on pixelsignals output from the image sensor; and a control circuit that sets animaging mode, in the image sensor, for alternately and repeatedlyexecuting the first imaging operation and the second imaging operation.6. The focus detection device of claim 5, wherein: the image sensorincludes a second imaging mode that performs a third imaging operationin which pixel signals corresponding to the pupil-division direction arecombined and a pixel signal for display is generated and output, and thecontrol circuit sets the imaging mode, for alternately and repeatedlyexecuting the first imaging operation and the second imaging operation,in the image sensor if still picture shooting is instructed when theimage sensor is repeatedly executing an imaging operation in the secondimaging mode.
 7. The focus detection device of claim 5, wherein: theimage sensor includes a third imaging mode for successively executingthe second imaging operation and the third imaging operation, and thecontrol circuit sets the imaging mode, for alternately and repeatedlyexecuting the first imaging operation and the second imaging operation,in the image sensor if still picture shooting is instructed when theimage sensor is repeatedly executing an imaging operation in the thirdimaging mode.
 8. A focus detection method, for a focus detection devicehaving an image sensor with a plurality of photo-diodes, arrangeddivided in a given pupil division direction, that generate a pixelsignal by subjecting respective light flux, that passes throughdifferent exit pupil regions of a optical system for a singlemicro-lens, to photoelectric conversion, comprising: when the imagesensor is executing a first imaging operation, performing processing tostore still image data based on pixel signals output from the imagesensor; when the image sensor is executing a second imaging operation,performing processing for focus detection based on pixel signals outputfrom the image sensor; and setting an imaging mode, in the image sensor,for alternately and repeatedly executing the first imaging operation andthe second imaging operation.
 9. The focus detection method of claim 8,wherein: the second imaging operation only outputs a pixel signalcorresponding to some pupil division directions of among the pluralityof photo-diodes.
 10. The focus detection method of claim 8, wherein: thesecond imaging operation outputs pixel signals corresponding to pupildivisions direction of all of the plurality of photo-diodes.
 11. Thefocus detection method of claim 8, wherein: the image sensor includes asecond imaging mode that performs a third imaging operation in whichpixel signals corresponding to the pupil-division direction are combinedand a pixel signal for display is generated and output, and furthercomprising setting the imaging mode, for alternately and repeatedlyexecuting the first imaging operation and the second imaging operation,in the image sensor if still picture shooting is instructed when theimage sensor is repeatedly executing an imaging operation in the secondimaging mode.
 12. The focus detection method of claim 11, wherein: in acase where an imaging mode, among the plurality of imaging modes, hasbeen set whereby the first imaging operation and the second imagingoperation are alternately and repeatedly executed, the first imagingoperation in executed before the second imaging operation.
 13. The focusdetection method of claim 8, wherein: the image sensor includes a thirdimaging mode for successively executing the second imaging operation andthe third imaging operation, and further comprising setting the imagingmode, for alternately and repeatedly executing the first imagingoperation and the second imaging operation, in the image sensor if stillpicture shooting is instructed when the image sensor is repeatedlyexecuting an imaging operation in the third imaging mode.
 14. The focusdetection method of claim 13, wherein: in a case where an imaging mode,among the plurality of imaging modes, has been set whereby the firstimaging operation and the second imaging operation are alternately andrepeatedly executed, the first imaging operation in executed before thesecond imaging operation.
 15. A non-transitory computer-readable mediumstoring a processor executable code, which when executed by at least oneprocessor, performs a focus adjusting method, the processor beingarranged within a focus detection device having an image sensor with aplurality of photo-diodes, arranged divided in a given pupil divisiondirection, that generate a pixel signal by subjecting respective lightflux, that passes through different exit pupil regions of an opticalsystem for a single micro-lens, to photoelectric conversion, the focusadjusting method comprising: when the image sensor is executing a firstimaging operation, performing processing to store still image data basedon pixel signals output from the image sensor; when the image sensor isexecuting a second imaging operation, performing processing for focusdetection based on pixel signals output from the image sensor; andsetting an imaging mode, in the image sensor, for alternately andrepeatedly executing the first imaging operation and the second imagingoperation.
 16. The storage medium of claim 15, wherein: the secondimaging operation only outputs a pixel signal corresponding to somepupil division directions of among the plurality of photo-diodes. 17.The storage medium of claim 15, wherein: the second imaging operationoutputs pixel signals corresponding to pupil divisions direction of allof the plurality of photo-diodes.
 18. The storage medium of claim 15,wherein: the image sensor includes a second imaging mode that performs athird imaging operation in which pixel signals corresponding to thepupil-division direction are combined and a pixel signal for display isgenerated and output, and the focus adjustment method further comprisessetting the imaging mode, for alternately and repeatedly executing thefirst imaging operation and the second imaging operation, in the imagesensor if still picture shooting is instructed when the image sensor isrepeatedly executing an imaging operation in the second imaging mode.19. The storage medium of claim 18, wherein: in a case where an imagingmode, among the plurality of imaging modes, has been set whereby thefirst imaging operation and the second imaging operation are alternatelyand repeatedly executed, the first imaging operation in executed beforethe second imaging operation.
 20. The storage medium of claim 15,wherein: the image sensor includes a third imaging mode for successivelyexecuting the second imaging operation and the third imaging operation,and the focus adjustment method further comprises setting the imagingmode, for alternately and repeatedly executing the first imagingoperation and the second imaging operation, in the image sensor if stillpicture shooting is instructed when the image sensor is repeatedlyexecuting an imaging operation in the third imaging mode.